A Performance Study of Two-Phase I/O

Massively parallel computers are increasingly being used to solve large, I/O intensive applications in many diierent elds. For such applications, the I/O subsystem represents a signiicant obstacle in the way of achieving good performance. While massively parallel architectures do, in general, provide parallel I/O hardware, this alone is not suucient to guarantee good performance. The problem is that in many applications each processor initiates many small I/O requests rather than fewer larger requests, resulting in signiicant performance penalties due to the high latency associated with I/O. However, it is often the case that in the aggregate the I/O requests are signiicantly fewer and larger. Two-phase I/O is a technique that captures and exploits this aggregate information to recombine I/O requests such that fewer and larger requests are generated, reducing latency and improving performance. In this paper, we describe our eeorts to obtain high performance using two-phase I/O. In particular, we describe our rst implementation which produced a sustained bandwidth of 78 MBytes per second, and discuss the steps taken to increase this bandwidth to 420 MBytes per second.